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Abstract This paper explores the influence of symbolic or representational learning materials on pupil 
engagement or learning outcomes, when 14-16 year old pupils use common types of science simulations. 
The project pilot phase involved three (15-16 year old) male pupils and a main phase involved txuenty- 
one (14-15 year old) pupils. A retrospective accounts methodology (Clarke, 1998) presented pupils with 
a digital record of their ‘think aloud ’ (Ericsson & Simon, 1984) behaviour with the simulation, and 
they were asked for retrospective comment. Pre- and post-surveys were also used. This interaction record 
for two boys is used to illustrate the findings. This record was chosen, because the boys spoke aloud 
throughout the period of engagement, which generated useful data for microanalysis. Findings suggest 
that pupils working with science simulations face a trans-disciplinary demand (computer competence, 
information processing skills, traditional language proficiency, and science understanding). In terms of 
traditional and subject literacy, pupils have to make sense of linguistically complex information when 
using common science simulations. They need to understand their subject, be confident and competent 
with the available technology, and possess language skills that enable them to establish links between the 
microscopic, macroscopic and sy mbolic components of science. 
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Introduction 

There is growing documentation on how computer-based technologies are 
impacting on communicative practices in school, and many are suggesting that this 
requires new understandings about literacy (Unsworth, 2001). Some (see 
Rodrigues, 2004) suggest that teachers need to use technologies as more than 
tools; they have to ensure that working with these technologies goes beyond simply 
locating and retrieving information. Users need to develop strategies that enable 
them to critically evaluate information and be able to work across semiotic systems 
in designing multimodal representations (Unsworth, 2001). It has been argued 
that literacy exists not only on a screen or on paper, but also in the practices that 
surround it and the situation in which literacy is earned out (Unsworth, 2001; 
Mackey, 2002). Lemke (1998) pointed out that science education, and scientific 
texts in particular, involve images (graphs, diagrams, etc) and written text, and that 
these components contribute differently to knowledge construction. Therefore, 
given the increasing push to use simulations in science, we need to understand how 
these different modalities separately and interactively help in the construction of 
meaning. 
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Many studies investigating changes in multimedia representation discuss the 
concept of working memory and cognitive load (Kester, Kirschner, Sc van 
Merrienboer, 2004; Kirriemuir Sc McFarlane, 2004), arguing that the nature of the 
representation (visual, text, aural, or various combinations) make significant cog¬ 
nitive demands on the user. For example, it has been posited that changing the 
presentation format in multimedia may reduce cognitive demand (Sweller, van 
Merrienboer, Sc Pass, 1998). Or to be more precise, as Mayer (2003) suggests, mul¬ 
timedia instructions are more effective, when aural and visual information is pre¬ 
sented close together (the contiguity principle), and instructions are effective 
when verbal information is presented aurally rather than visually (the modality 
principle). However, Tabbers, Martens, and van Merrienboer, (2004) suggest that 
the reduction in cognitive load findings could also be explained by a reduction in 
visual search. In addition, the cognitive load argument finds itself in a quandary, 
because similarly demanding representations that are commonly found in com¬ 
puter games do not appear to generate cognitive overload. 

Regardless of these cognitive load issues, the last two decades have seen an 
increase in the number of calls to include various forms of multimedia and com¬ 
puter-based technology in science education (see Rodrigues, 2004). Support for 
the integration of computer-based technology is guided by a belief that use will 
empower teachers and pupils, and result in enhanced learning outcomes (Leach 
Sc Moon, 2000; Loveless Sc Ellis, 2001). However, if digital literacy involves the abi¬ 
lity to use computer-presented information in multiple formats from a wide range 
of sources (Glister, 1997), then informed use of multimedia in science will require 
digital literacy. Though the concept of digital literacy is still developing, the 
National Research Council: Committee on Information Technology Literacy 
(1999) considers digital literacy to be an ability to apply information technology in 
complex and sustained situations, and to understand the consequences of the 
action. Digital literacy in science education can therefore be viewed as the rela¬ 
tionship between process/skills-based components and concept-based components 
that are particular to the science classroom. The process/skill-based component 
includes computer skills (technical computer related competence) and informa¬ 
tion processing skills (a competence that requires being able to recognize infor¬ 
mation need, retrieve, evaluate, use and disseminate information). The concept- 
based component includes traditional literacy (language competence) and subject 
literacy (science competence). A small-scale project involving 24 students from two 
schools was initiated in order to explore the influence of these components on the 
way students make sense of science. 

Methodology 

The project involved two phases. The first phase piloted the technique with 
three male students (aged 15-16 years). A retrospective accounts methodology 
(Clarke, 1998) involving ‘thinking aloud’ (Ericsson Sc Simon, 1984) was used to 
access students’ thinking. The ‘thinking aloud’ strategy has been used in cognitive 
psychology research, and is often used to explore pupils’ problem-solving strate¬ 
gies; however, recent times have seen this strategy used to explore pupil-computer 
interactions (Crowther, Keller, Sc Waddoups, 2004). Digital records documenting 
students’ ‘at-the time’ behaviour and actions were collected and then replayed to 
them. The students were asked to explain their actions during this informal, semi- 
structured retrospective interview. In addition, the students completed pre and 
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post surveys. The main phase involved 21 volunteers (male and female students 
aged 14-15 years) from another school. These students worked in pairs (apart from 
one group of three). The paired approach was an attempt to reduce students’ view¬ 
ing this task as a test, and it was felt that pair working may encourage more ‘think¬ 
ing aloud’ as students conversed. Digital records were collected and then replayed 
during the semi structured interviews. The transcripts found in this paper have 
been coded to render the students some degree of anonymity. 

Analysis of one record is reported in this paper. It was chosen because the two 
male students were talking aloud throughout the whole period of engagement, 
and their digital record therefore provides useful data for microanalysis. This digi¬ 
tal record and transcript was made available to a team of four comprising the prin¬ 
cipal investigator, a language specialist, a science specialist, and a technology spe¬ 
cialist. Using a modified version of Clarke’s (1998) complementary accounts 
methodology approach, the insights provided by these four perspectives inform 
the discussion in this paper. 

This article reflects on the engagement with a simulation that required stu¬ 
dents to make sense of multiple format information. Figure 1 provides a screenshot 
of the simulation under discussion. It is a commonly found type of science simula¬ 
tion. 



Figure 1. A screenshot to a Simulation 


The simulation used in this project is freely available on the following website, 

http://www.chem.iastate.edu/group/Greenbowe/sections/projectfolder/anima- 

tionsindex.htm . It is typical of many commercially available science education sim¬ 
ulations. The website provided tutorial worksheets, but these were not used, as the 
intention was to gain an insight into the digital literacy demands made by the sim¬ 
ulation and not the support documentation. 

The pupils described in this paper use a simulation representing the reaction 
between sodium hydroxide and hydrochloric acid, on a microscopic level. The si¬ 
mulation shows a series of tab instructions on the top edge of the simulation, some 
text, some icons and some symbols on screen. If the simulation is allowed to run its 
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course, the icons and symbols move erratically and collide sometimes, and occa¬ 
sionally new text appears on screen. This text either provides direction regarding 
continued use of the simulation or information relevant to making sense of the sci¬ 
ence. 


Findings 

The digital recordings of the pupils working together with the simulation sug¬ 
gest that they are familiar with working together. For example, they complete each 
other’s sentences: 

Sam They are like (pause). Right. So., basically, we have got all these wee... 

Jim Things... 

Sam Negatives. Neutrons and . 

Jim Charge things ... 

Sam Protons and ... 

Jim Neutrons ... 

Sam Electrons and protons and.. 

Jim Neutrons 

Transcript 1. Excerpt from Pupil Engagement Recording 

Their discussions are purposeful. Their register also suggests that they are com¬ 
fortable working together and the change in register whenever they reach a con¬ 
clusion suggests that they are confident. 


Technology 

The two male pupils will be referred to in this paper as Jim and Sam. They were 
asked to rate their experience level and comfort level for several applications, using 
a scale of 1-4. where the ratings indicated the following:: 

Experience level (E) Comfort level (C) 

1= a lot of experience 1= very comfortable 

2= some experience 2= moderately comfortable 

3=a little experience 3= would need some help to feel comfortable 

4 = no experience 4 would need lots of help to feel comfortable 

Table 1 documents their self-rating, their experience rating (E) value is pre¬ 
sented first, and then their comfort level (C) rating. 


Table 1 


Comfort and Experience with Various Technologies. 


Activities using technology 

Jim 



Sam 


Experience 

Comfort 

Experience 

Comfort 

Word processing 

1 

1 

1 

1 

Databases 

2 

2 

3 

3 

Presentation software/Powerpoint 

2 

2 

1 

1 

Computer games 

1 

1 

1 

1 

Internet search engines 

1 

1 

1 

1 

Simulations 

3 

3 

3 

2 

Desktop publishing 

3 

3 

3 

3 
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Jim (aged 14) and Sam (aged 15) felt they were very or moderately comfort¬ 
able with word processing and playing computer games, but not very or moderately 
comfortable using simulations. These experience and comfort ratings are of inter¬ 
est for two reasons. First, it would appear that these two pupils do not equate com¬ 
puter games with simulations, and second, their Internet use does not predomi¬ 
nantly involve simulations. These two pupils were part of the project sample of 24. 
Overall, 16 of the 24 pupils felt that they had lots of experience of word processors, 
17/24 said they had lots of experience of computer games, and 17/24 said they 
had lots of experience of the internet, but only 4 said they had lots of experience 
with simulations. 

The simulation is neither as complex nor as vivid as that found in the average 
computer game. In essence, information is presented through dialogue boxes and 
two-dimensional dynamic icons, with a pale blue screen background. Yet, in gene¬ 
ral the dialogue between Jim and Sam suggests that they are engaged when using 
the simulation. 

From a technology literacy perspective, although the visual media is not as 
sophisticated as that encountered in many computer games, the need to pay atten¬ 
tion to detail is significant. The tab buttons provide access to the technology com¬ 
ponent of the simulation, i.e., the need to replay, or revisit the animation is mana¬ 
ged by using these tab buttons. In addition, the introduction of labelled icons, 
albeit fleetingly in some cases, provides some clues to help make sense of the sci¬ 
ence. The text boxes provide access to key words and useful instruction. These on 
screen text boxes help situate the animation in terms of the science. 

The labels that accompany the icons are in some cases fleeting, while others 
remain permanently on screen. For example, the spheres representing NaOH are 
initially introduced with that label. Likewise, the sphere representing Cl- is also 
labelled. However, on dissociation, the Na+ is reduced to being represented as a 
sphere with ‘+\ Likewise, the Cl- ion is reduced to being represented as a sphere 
with a alongside. These subliminal clues resulted in the pupils associating the 
simulation with atomic charge (protons, neutrons, and electrons). The initial text 
box that signalled the simulation as a neutralisation experiment between acid and 
alkali was on screen at the start, and while it was accessible whenever the pupils 
replayed the simulation, it was not ‘on screen’ to the same extent as the spheres 
with positive or negative symbols. In addition, the text boxes with information rel¬ 
evant to science rather than activity instruction may have been better presented 
through a sound track. 

Linguistics 

Language is considered a crucial element in the understanding of learning 
through negotiated meaning (Holliday & Hessian, 1980) and indeed in the wider 
mechanism of thought (Vygotsky, 1934/1987). The boys’ language can be inter¬ 
preted in terms of pedagogical ‘moves,’ such as ‘structuring’ ‘soliciting’ ‘respond¬ 
ing,’ and ‘reacting’ (Sinclair & Coulthard, 1975). Structuring is where the learning 
situation is established, soliciting usually takes the form of a question or request, 
reacting involves modification and shaping of moves, and responding tends to be 
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represented as interactive conversations. In addition to interpreting talk in terms 
of pedagogical moves, systemic functional linguistics can be used to help describe 
the dimensions of the interaction (Christie & Unsworth, 2000) in terms of the 
‘Field’ (what happens in that situation), ‘Tenor’ (the personal - power relations) 
and ‘Mode’ (the part that language plays in learning and securing relationships). 

Analysis of the register, while the two pupils were using the simulation, suggests 
that the Tenor is equitable. The two boys share ideas, their pedagogical moves show 
equitable turn taking in structuring, soliciting, responding, and reacting. The Field 
that evolves shows the two boys making progress with respect to technical compe¬ 
tence in managing the simulation. Their dialogue suggests that they hold similar 
views with respect to understanding the simulation, but this element of Mode will 
be considered in more detail in the following section on science. 

The boys’ register also suggests that they are confident with their interpreta¬ 
tion of the simulation, though their soliciting and responding statements suggest 
they are still dying to make sense of particular elements. Although Sam’s register 
suggests more confidence, Jim is more active in and this suggests that Jim is more 
involved in structuring. Fie directs Sam’s attention and puts forward ideas for con¬ 
sideration, while Sam tends to accept challenges to his thinking, but, in general, 
they negotiate activity direction. 

Sam Very interesting. 

Jim Wait, did all those minuses just become like neutral or something? 

Sam I have no idea. 

Jim. Watch. 

Sam See, like right at the start 

Jim Look at that positive, right. 

Sam See if it changes into a green one or something 

Jim See watch these ones coming in (points to screen) 

Sam That just lost it’s minus. 

Jim Aye, It got a plus from there. 

Transcript 2. Excerpt from Pupil Engagement Recording 

Although the text on screen was limited, the pupils paid scant attention to key 
information that needed to be read. Having read the text box, the pupils were then 
required to translate and transform this language into the language of science in 
terms of icons and symbols. Consequently, the pupils interpreted the word ‘neu¬ 
tralization’ in terms of the reaction between protons and electrons rather than the 
reaction between an acid and an alkali. 

Science 

The survey data shows their familiarity with areas of chemistry as it is indicated 
in Table 2. 
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Table 2 

Comfort and Experience with Various Chemistry Topics 


Chemistry topic 

Jim 

Sam 



Experience Comfort 

Experience 

Comfort 

Atoms and Molecules 

2,2 

2 

2 

Balancing equations 

2,2 

2 

2 

Acids reacting with alkalis 

2,2 

1 

2 

Titration experiments 

3,3 

3 

3 

Reactions of metals 

2,2 

2 

2 

Conductivity 7 of Solutions 

2,2 

1 

1 

Dissolving 

2,2 

2 

2 


Analysis in terms of ‘Mode’ also helps in the analysis of the pupils’ under¬ 
standing of the science concepts they encounter. The simulation involved the use 
of every day language, science terminology, science symbols, and iconic represen¬ 
tations. For example, the introductory screen contained a text box that included 
everyday language and science terminology (such as neutralization), it also con¬ 
tained tab buttons with text instructions (such as continue), and it contained col¬ 
ored spheres intended to represent molecules and ions, and symbols to identify 
these molecules and ions. 


There is a notable difference in the two pupils’ science literacy. 


Sam 

You have to replay it. (Reads the text box, aloud) The following is a rep¬ 
resentation of a neutralisation when NaOH is added to aqueous H C i 

Jim 

H. C. L. 

Sam 

Are you sure that is an L? 

Transcript 3. Excerpt from Pupil Engagement Recording 

Analysis of the transcript and the post use narrative account, suggests that Jim 

was able to interpret various sources made available through the simulation. He ini- 

daily shared Sam’: 

s interpretation of the simulation, but his post-use narrative 

account suggests that he w^as also reserving judgment and considering whether the 
simulation represented ions, whereas Sam was still interpreting the simulation 
from the atomic model perspective. 

Interviewer 

What was becoming neutral? 

Sam 

The protons. 

Jim 

And the minus ones. 

Sam 

Electrons. 

Interviewer 

Ok, So, what are the green things then ? 

Sam 

Neutrons. I mean electrons. 

Interviewer 

Ok. If I told you that this is .... 

Jim 

Ions. 

Interviewer 

Yeah. Would that make any difference? 

Jim 

Yes, because they are like charged things. 

Sam 

Are ions like when... 

Jim 

Charged particles. 


Transcript 4. Excerpt from Post Use Narrative. 
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Information Processing Skills 

Though the dialogue suggests that the two boys are working well together, 
analysis of the transcript and the retrospective narrative suggest that they are 
attending to different elements of the simulation. Hence, their ‘Mode’ (the part 
that language plays in learning and securing relationships) is different. For exam¬ 
ple, in the following clip, Jim identifies the introduction of something (italicized in 
the transcript), and though Sam continues to describe the activity of the other 
spheres, Jim’s transcript suggests he is more observant with respect to the finer 
detail of the simulation, and hence paying more attention to a variety of symbols 
and text. 

Sam They are really hitting off the xvalls and each other and stuff It's quite men¬ 
tal. . .press continue. 

Jim Something is added. 

Sam They are like hitting each other, greens and reds and purples. Some of them are 
pluses. Oh the green ones are minuses. 

Jim Purple ones are pluses. 

Sam And the red ones are in-between, so Tm guessing that the green, the red ones are, 
ok now this is weird the green ones are obviously neutrons. 

Jim No they re minus. 

Sam Electrons, that's what I mean. Electrons are the minus ones. 

Jim NaOH is added. 

Transcript 5. Excerpt from Pupil Engagement Recording 

Discussion 

Students working in an e-science learning environment face a trans-disciplinary 
demand. The influence of computer competence, information processing skills, 
traditional language proficiency, and science understanding levels in e-science 
learning environments should not be underestimated. In terms of traditional and 
subject literacy, pupils have to make sense of linguistically complex information, in 
effect use higher order information processing skills, when using common science 
simulations. They need to understand their subject, be confident and competent 
with respect to using the available technology, and have language skills that enable 
them to establish links between the microscopic, macroscopic, and symbolic com¬ 
ponents of science. 

Of course, the context in which the simulation is deployed will have a signifi¬ 
cant impact on the way it is used. However, the intention of this project was to inve¬ 
stigate the trans-disciplinary demand pupils faced when they used commonly 
found simulations. It could be argued that the insight provided by using a small 
group does not lend itself to generalisations with respect to trans-disciplinary 
demands. Indeed, it could be argued that the findings could be specific to the boys 
described in this paper. However, it should be noted that the pupils’ talk as pre¬ 
sented in this paper was representative of the project cohort. Further illustrations 
of pupil talk can be found in Rodrigues (2007), where individual differences 
among the pupils using a variety of simulations are used to illustrate the e-literacy 
skills and demands. Exploring the use of the simulation in a classroom context, 
with the teacher providing support, would not have enabled us to investigate the 
demands made by the simulation, as the teachers’ strategies in deploying the sim¬ 
ulation would have influenced pupils’ engagement. In addition, the rhetoric sur- 
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rounding the use of technology in science often alludes to it providing an oppor¬ 
tunity for independent learning and student ownership of the learning process. 

The fact that reports (see for example, Smeets 8c Mooij, 2001; Zhao &Cziko, 
2001) continue to signal that practice involving the use of various technologies has 
not changed significantly, also suggests the development of digital literacy in e-sci- 
ence education environments is most likely incidental. Digital literacy needs to be 
taught more explicitly, if the available technology is to do more than simply pro¬ 
vide increased amounts of better-presented data. 

Recommendations for designers 

• Although the pupils in this project were interested and engaged with the 
simulation, it was lacklustre in terms of presentation in comparison with 
computer games. The gap in sophistication and presentation between 
computer games and educational software needs to be addressed. 

• Review how pupils will interpret the on going presence or removal of par¬ 
ticular symbols or icons. Include checks that pupils need to complete to 
ensure that they have made the associations intended by the designer. 

• Consider establishing explicit links between visual text and symbols or 
icons. 

• Use a variety of media to support pupils’ understanding. Include oral nar¬ 
ratives and keep text information related to science on screen. 

Recommendations for users 

• The pupils need to be taught to read screens for informing and for learn¬ 
ing. They need to be taught how to retrieve information from any given 
screen shot. 

• If the pupils had not mis-identified sodium hydroxide, they may well have 
reconsidered their interpretation of the simulation. They need to be 
taught to verify and confirm their interpretation of key information before 
they reach firm conclusions. 

• They need to be taught information processing skills. 

Just as many have long argued that teacher discourse needs to be unambigu¬ 
ous, relevant and coherent, so now, science simulations also need to ensure that 
they are unambiguous, relevant and coherent from the user’s perspective. 

References 

Christie, F., 8c Unsworth, L. (2000) Developing socially responsible language 
research, pl-26, In L. Unsworth (Ed) Researching Language in Schools and 
Communities. London: Cassell. 

Clarke, D. J. (1998). Studying the classroom negotiation of meaning: 
Complementary accounts methodology. (Chapter 7). In A. Teppo (Ed.) 
Qualitative research methods in mathematics education. Monograph Number 9 of 
the Journal for Research in Mathematics Education. Res ton, VA: NCTM, pp. 98- 
111 (ISBN 0-87353-459-X) 

Crowther, M. S., Keller, C. C., 8c Waddoups, G. L. (2004). Improving the quality 
and effectiveness of computer-mediated instruction through usability evalu¬ 
ations. British Journal of Educational Technology , 35(3), 289-304. 

Ericsson, K. A., 8c Simon, H. A. (1984), Protocol analysis: Verbal reports as data 
(Revised ed.). London: MIT Press. 


14 Joe McLuckie, Susan Rodrigues, Neil Taylor, and Graham Williamson 


Glister, P. (1997). Digital literacy. New York: Wiley and Computer Publishing 

HaLLTDAY, M. A. C., Sc Hassan, R (1980) Language, Context and Text: Aspects of 
Language in a Social Semiotic perspective Melbourne: Deakin University Press 

Rester, L., Kirschner, P. A., & van Merrienboer, J. J.G. (2004). Information pre¬ 
sentation and trouble shooting in electrical circuits, International Journal of 
Science Education, 26(2), 239-256. 

Kirriemuir, J., Sc McFarlane, A. (2004). Report 8: Literature review in games learn¬ 
ing. A report for NESTA futurelab. http://www.nestafuturelab.org/ 
research/reviews/08 01.htm Retrieved December 2005. 

Leach, J., Sc MOON, J. (2000). Pedagogy, information and communications. The 
Curriculum Journal, 11, 385-404. 

Lemke, J. L. (1998). Metamedia literacy: transforming meanings and media, In D 
Reinking, M. C., McKenna, L. Labbo, and R. D Kieffer (Eds) Handbook of 
Literacy and technology: Transformations in a post typographic world (pp. 282-302). 
Mahway, N.J: Erlbaum. 

Loveless, A., Sc Ellis, V. (Eds) (2001). ICT pedagogy and the curriculum: subject to 
change. London: Routledge Falmer. 

Mackey, M (2002). Literacies across Media: Playing the text. London: Routledge. 

Mayer, R. E. (2003). The promise of multimedia learning: Using the same instruc¬ 
tional design methods across different media. Learning and Instruction 
13(2), 125-140. 

National Research Council: Committee on Information Technology Literacy 
(1999). being fluent with information technology. Washington, DC: 
National Academy Press, http://stills.nap.edu/html/beingfluent Retrieved 
15^ December 2005. 

Rodrigues, S. (2004). A review of student use of ICT in science: Comparing the lit¬ 
erature rhetoric with learning reality. Paper presented at the IOSTE sympo¬ 
sium, Lublin, Poland, 29 th July 2004. 

Rodrigues, S. (2007) Factors that influence pupil engagement with science simu¬ 
lations: the role of distraction, vividness, logic, instruction and prior knowl¬ 
edge, Chemical Education Research and Practice, 8(1), 1-12 

Sinclair, J., McH, Sc Coulthard, R. M. (1975) Towards and analysis of Discourse: The 
Englsih used by teachers and pupils. London: Oxford University Press 

Smeets, E., Sc Mooij, T. (2001). Pupils centred learning, ICT and teacher behav¬ 
iour: observations in educational practice, British Journal of Educational 
Technology, 32(4), 403-417. 

Sweller, J., van Merrienboer, J.J. G., Sc Pass, F. (1998). Cognitive architecture and 
instructional design Educational Psychology Review, 10, 251-296. 

Tarbers, H. K., Martens, R. L., Sc van Merrienboer, J. J.G. (2004). Multimedia 
instructions and cognitive load theory: effects of modality and cueing, British 
Journal of Educational Psychology, 74, 71-81. 

Unsworth, L (2001). Teaching Multiliteracies across the curriculum. Buckingham: 
Open University Press 

Vygotsky, L. S. (1934/1987) Thought and Language. Cambridge, MA: MIT press. 

Zhao Y., Sc Cziko, G. A. (2001). Teacher adoption of technology: A perceptual con¬ 
trol theory perspective. Journal of Technology and Teacher Education, 9(19), 5-30. 




